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integrating the multimedia object descriptions and the entity relation graph descriptions 
to generate at least one description record to represent content embedded within the multimedia 
content. 



0> 



26. (New) The method of claim 25, further comprising generating, from the multimedia 
object descriptions, multimedia object hierarchy descriptions by object hierarchy construction 
and extraction processing, for at least one of the multimedia types. 

27. (New) The method of claim 25, wherein the multimedia types include at least one of 
image, audio, video, synthetic, and text. 

28. (New) The method of claim 25, wherein the extracting of multimedia objects further 
comprises: 

segmenting each multimedia content into segments including content from at least one of 
the multimedia types for the multimedia content; and 

generating at least one feature description for at least one of the segments by feature 
extraction and annotation; 

wherein the generated multimedia object descriptions comprise the at least one feature 
description for the at least one segment. 

29. (New) The method of claim 28, wherein the segments are selected from the group 
consisting of local segments and global segments. 



30. (New) The method of claim 28, further comprising selecting the at least one feature 
description from the group consisting of media, semantic and temporal features. 

3 1 . (New) The method of claim 30, wherein the media features are further defined by at least 
one feature description selected from the group consisting of data location, scalable 
representation and modality transcoding. 
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32. (New) The method of claim 30, wherein the semantic features are further defined by at 
least one feature description selected from the group consisting of keywords, who, what object, 
what action, why, when, where and text annotation. 



33. (New) The method of claim 30, wherein the temporal features are further defined by at 
least one feature description consisting of duration. 



\ 



34. (New) The method of claim 28, wherein the extracting of multimedia objects further 
comprises: 

generating media object descriptions from the multimedia segment for one of the 
multimedia types by media object extraction processing; 

generating media object hierarchy descriptions from the generated media object 
descriptions by object hierarchy construction and extraction processing; and 

generating media entity relation graph descriptions from the generated media object 
descriptions by entity relation graph generation processing. 

35. (New) The method of claim 34, wherein generating media object descriptions further 
comprises: 

segmenting the content of each multimedia type in the multimedia object into segments 
within the multimedia object by media segmentation processing; and 

generating at least one feature description for at least one of the segments by feature 
extraction and annotation; 

wherein the generated media object descriptions comprise the at least one feature 
description for the at least one of the segments. 

36. (New) The method of claim 35, further comprising the step of selecting the at least one 
feature description from the group consisting of media, semantic and temporal. 



37. (New) The method of claim 35, wherein generating media object hierarchy descriptions 
generates media object hierarchy descriptions of the media object descriptions based on media 
feature relationships of media objects represented by the media object descriptions. 
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38. (New) The method of claim 35, wherein generating media object hierarchy descriptions 
generates semantic object hierarchy descriptions of the media object descriptions based on 
semantic feature relationships of media objects represented by the media object descriptions. 

39. (New) The method of claim 35 5 wherein generating media object hierarchy descriptions 
generates temporal object hierarchy descriptions of the media object descriptions based on 
temporal feature relationships of media objects represented by the media object descriptions. 

40. (New) The method of claim 35, wherein generating media object hierarchy descriptions 
generates media object hierarchy descriptions of the media object descriptions based on 
relationships of media objects represented by the media object descriptions, and wherein the 
relationships are selected from the group consisting of media feature relationships, semantic 
feature relationships, temporal feature relationships, and spatial feature relationships. 

41 . (New) The method of claim 35, wherein generating media entity relation graph 
descriptions generates entity relation graph descriptions of the media object descriptions based 
on relationships of media objects represented by the media object descriptions, wherein the 
relationships are selected from the group consisting of media feature relationships, semantic 
feature relationships, temporal feature relationships and spatial feature relationships. 

42. (New) The method of claim 26, wherein generating multimedia object hierarchy 
descriptions generates multimedia object hierarchy descriptions of the multimedia object 
descriptions based on media feature relationships of multimedia objects represented by the 
multimedia object descriptions. 

43. (New) The method of claim 26, wherein generating multimedia object hierarchy 
descriptions generates semantic object hierarchy descriptions of the multimedia object 
descriptions based on semantic feature relationships of multimedia objects represented by the 
multimedia object descriptions. 
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44. (New) The method of claim 26, wherein generating multimedia object hierarchy 
descriptions generates temporal object hierarchy descriptions of the multimedia object 
descriptions based on temporal feature relationships of multimedia objects represented by the 
multimedia object descriptions. 



45. (New) The method of claim 26, wherein generating multimedia object hierarchy 
descriptions generates multimedia object hierarchy descriptions of the multimedia object 
descriptions based on relationships of multimedia objects represented by the multimedia object 
descriptions, wherein the relationships are selected from the group consisting of media feature 
relationships, semantic feature relationships, temporal feature relationships and spatial feature 
relationships. 

46. (New) The method of claim 25, wherein generating entity relation graph descriptions 
generates the entity relation graph descriptions of the multimedia object descriptions based on 
relationships of multimedia objects represented by the multimedia object descriptions, wherein 
the relationships are selected from the group consisting of media feature relationships, semantic 
feature relationships, temporal feature relationships and spatial feature relationships. 

47. (New) The method of claim 25, further comprising receiving and encoding the 
multimedia object descriptions into encoded description information, and storing the encoded 
description information as the at least one description record. 

48. (New) The method of claim 26, further comprising combining the multimedia object 
description, the multimedia object hierarchy descriptions, and the entity relation graph 
description to form a multimedia description, and receiving and encoding the multimedia 
description into encoded description information, and storing the encoded description 
information as the at least one description record. 

49. (New) The method of claim 47, wherein the encoding comprises binary encoding. 

50. (New) The method of claim 48, wherein the encoding comprises binary encoding. 
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